Lecture 13A: From Notebooks to the Web: Github Pages, Web Servers, and Dash

November 24, 2020

Housekeeping

Final project due: 5pm on Monday, December 21st

Outline for the rest of the course

We'll discuss ways to translate our analysis results to the Web in a meaningful way. We'll cover three methods to do so, each with their own pros and cons:

  1. Embedding interactive charts on the Web via Github Pages
  2. Creating Web apps & dashboards using the Python library Dash
  3. Creating Web apps & dashboards using the Python library Panel

Today: we'll focus on the first two today and cover Panel in detail in week 14

Part 1: Embedding Interactive Charts on the Web

To start, let's recap three ways that we've learned to produce interactive charts in the course:

Example 1: Measles Incidence in Altair

Load the data from week 2:

Use the pandas.melt() function to convert it to tidy format:

Now let's load altair:

Saving Altair Plots

Altair plots can be fully represented as JSON data. This makes them very easy to embed on websites, as we shall soon see!

Now, let's compare the HTML and JSON files...

Measles Incidence in hvplot

Saving Hvplot Plots

HTML is are only option here...

Example 3: Folium + OSMnx

Identify the lat/lng coordinates for our places of interest: Use osmnx to download the geometries for the Libery Bell and Art Museum

Get the street graph in Center City: Use osmnx to download the street network around City Hall.

Identify the nodes in the graph closest to our points of interest.

Use networkx to find the shortest path

Saving Folium Maps

Just use the save() function!

How can we display these charts on the Web?

We can embed them on GitHub Pages...

Part 2: Github Pages

Template Site with Examples

Github Pages URL

The structure of the URL for the rendered page is:

https://[USERNAME].github.io/[REPOSITORY NAME]

Note: you can create a new website for every repository, so this will work even if you have a personal Github Pages website set up.

Step 1: Create your own repository


For more information, see this guide on creating a repository from a template.

Step 2: Choose a name and description

Step 3: Enable Github Pages on your new repository

On the home page for your new repository go to "Settings":

Step 4: Customize your site

For more information on the config file, see the documentation.

Step 5: Publish a post

Adding new posts

To add new posts, simply add a file in the _posts directory that:

  1. follows the convention YYYY-MM-DD-name-of-post.ext
  2. Includes the necessary header material (see the next slide)

You can take a look at the source for the existing posts in the _posts folder to get an idea about how it works.

The anatomy of a post

Source: https://raw.githubusercontent.com/MUSA-550-Fall-2020/github-pages-starter/master/_posts/2019-04-17-example-post.md

Rendered: https://musa-550-fall-2020.github.io/github-pages-starter/example-post/

How can we embed our (static) Matplotlib charts?

These are just normal PNG images — we can use Markdown's syntax for embedding images.

Embedding images: the syntax

![alt-text]({{ site.url }}{{ site.baseurl }}/assets/images/YOUR_IMAGE_FILE.png

Steps

  1. Place your image in the assets/images/ folder
  2. Change YOUR_IMAGE_FILE.png to the name of your image and leave the rest of the path unchanged.

Note: the curly brackets for site.url and site.baseurl are template variables. When the site is rendered, these variables automatically get filled in so that the absolute path to the PNG file is correct.

How can we embed our interactive charts?

In the header, we can specify the charts to load using special loaders, which can handle three different types of files:

Embedding Altair/Hvplot charts: example

Source: https://raw.githubusercontent.com/MUSA-550-Fall-2020/github-pages-starter/master/_posts/2019-04-13-measles-charts.md

Rendered: https://musa-550-fall-2020.github.io/github-pages-starter/measles-charts/

Embedding Altair/Hvplot charts: the header

Header syntax

Under the altair-loader and hv-loader we need to specify two things:

  1. The CSS "id" attribute for the div where we want the chart to go
  2. The path to the HTML or JSON file to load (must be in the charts/ folder)

This should be specified as key: value pairs, for example:

altair-chart-1: "charts/measlesAltair.json"

In this case altair-chart-1 is the CSS identifier and "charts/measlesAltair.json" is the name of the file to load.

Note: there must be a matching "div" element with the same CSS identifier — this is where the chart will be embedded!

Important

Embedding Altair/Hvplot charts: the content

Embedding Folium charts: example

Source: https://raw.githubusercontent.com/MUSA-550-Fall-2020/github-pages-starter/master/_posts/2019-04-13-folium-charts.md

Rendered: https://musa-550-fall-2020.github.io/github-pages-starter/folium-charts/

Embedding Folium charts: the header

Header syntax

For the folium-loader, the syntax is:

CSS identifier for chart : ["chart file path", "width of chart in pixels"]

Again, there must be a matching "div" element with the same CSS identifier — this is where the chart will be embedded!

Embedding Folium charts: the content

So, which files are important?

There's a lot of extra stuff we don't actually need. The most important files/folders are:

Exercise: Setup your own Github Pages project page

Steps:

Notes

Part 3: Web Servers

A few quick basics before we move on to interactive Web apps with Dash...

Some (oversimplified) basics

Two main types:

Static and dynamic

Static Web Servers

Dynamic Web Servers

For more information: Mozilla documentation

CARTO: a dynamic web server

Anatomy of a request

Flask: a micro web framework in Python

A very lightweight package for framework for dynamic web apps

Flask documentation

Example 1: your first Flask app

Getting set up: launch a terminal in Jupyer notebook

Kapture%202020-11-22%20at%2016.30.27.gif

Steps:

From within the Jupyter Notebook Dashboard:

Flask code: hello.py

Important: this won't work on Binder, it needs to be running on your local machine

To start a local server

Run the following commands from the Terminal window in Jupyter notebook:

cd hello-flask
python hello.py

The "python" command

StackOverflow answer about the if-name-main syntax in Python.

Note

There are sometimes Anaconda-related issues where "python" will point to the wrong version of Python. You can double check by running:

which python

from the Jupyter Terminal interface. This should print out a path name that includes the name of our Python environment (musa-620)

Take a look at hello.py

View your application!

Navigate in the browser to: http://0.0.0.0:5000

You should see the text: "Hello, World!"

Important: Windows Users

You will likely need to navigate to localhost:5000 instead of http://0.0.0.0:5000

Note: the web address http://0.0.0.0 is an alias for "localhost" — this means the web server is running locally on your computer.

Editing your app with a running server

Try editing the hello.py file in your editor, saving, and re-loading the application page. You should see the app update to reflect the changes!

Example 2: using templates

Flask code: hello-template.py

From the terminal window

python hello-template.py

The main page ("/") renders the same thing as the previous app ("Hello, World!").

Navigate to the "/hello/" route and you should see:

Example 3: setting up an API with request parameters

hello-api.py: a more sophisticated example using *request parameters**

Run:

python hello-api.py

Flask code: hello-api.py

This uses default request parameters: days = 90 and fatal = 0

Now pass "days" and "fatal" parameters

This returns the number of fatal/nonfatal shootings in the past X "days"

Getting closer to a more realistic use case...

Steps:

  1. Get request parameters from the browser (input by the user)
  2. Query CARTO database for data based on that input
  3. Perform an operation on that data (in this case, count number of shootings)
  4. Return computed result back to the browser

Note: If you receive a "OSError: Address already in use"

Now, that we have the basics on to a more advanced library...

Part 4: Dash

The result of Python programmers asking the question: can we build a dashboard just in Python?

The answer is yes...for the most part.

You still need to use some CSS styling and know about different HTML elements. But everything is coded in Python.

Dash

Benefits

Downsides

The general workflow

In your Python Dash app, there are two main steps:

  1. Define the HTML layout, e.g., "div" elements, slider elements, etc.
  2. Define functions that take inputs from the interactive widgets from Step 1. and return the output for other HTML elements.
    • When the user changes a slider, dropdown, etc, the function will run, and return the updated output

Dash HTML components = "the app layout"

See: https://dash.plot.ly/dash-html-components

Dash HTML Example

html.Div([
    html.H1('Hello Dash'),
    html.Div([
        html.P('Dash converts Python classes into HTML'),
        html.P('This conversion happens behind the scenes by Dash's JavaScript front-end')
    ])
])

This gets converted automatically to the following HTML in your web app:

<div>
    <h1>Hello Dash</h1>
    <div>
        <p>Dash converts Python classes into HTML</p>
        <p>This conversion happens behind the scenes by Dash's JavaScript front-end</p>
    </div>
</div>

Dash core components = "widgets"

See: https://dash.plot.ly/dash-core-components

Note: Markdown is also supported!

Use the dcc.Markdown() object to automatically convert Markdown into HTML in your web apps!

Reference: HTML and CSS tutorials

The core components and HTML components in Dash remove most of the direct HTML/CSS, but still good to know some of the basics:

Useful to keep these references handy if you are unsure about syntax for HTML and/or CSS

The Dash Getting Started Guide

https://dash.plot.ly/getting-started

Combining Dash with Altair and Folium

Key: any valid ".html" block can be embedded within an "IFrame" element, as we have seen

Two key elements of Dash apps

Steps:

  1. Define a IFrame() element as part of the layout
  2. Given some user input, generate our Altair charts and save them as HTML
  3. Assign the chart HTML to the IFrame on our page

Let's see some Dash + Altair examples...

Two examples in the "dash-altair/" folder...

Visualizing the "penguins" dataset

Just like with Flask, we can run from the Jupyter terminal:

python dash_altair_penguins.py

Dash app

Translating the shootings app to Dash

We can run from the Jupyter terminal:

python dash_altair_shootings.py

Dash app

More information: Dash user guide

One more Dash example: Dash + Folium

From the base repository folder, run

cd dash-folium
python app.py

to start the example

Deploying a Dash app

Getting started

Launching your app on Heroku

Summary: web-based visualizations so far

Next Week: Panel

Exercise: modifying the Dash shootings apps

Let's modify either the Altair-based or Folium-based Dash app for the shootings data in Philadelphia to include a dropdown that allows the user to specify which race to show data for.

Steps